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Abstract. Symbolic approaches to the control design over complex systems employ the construction of finite- 
state models that are related to the original control systems, then use techniques from finite-state synthesis 
to compute controllers satisfying specifications given in a temporal logic, and finally translate the synthesized 
schemes back as controllers for the concrete complex systems. Such approaches have been successfully devel- 
^ oped and implemented for the synthesis of controllers over non-probabilistic control systems. In this paper, we 

extend the technique to probabilistic control systems modeled by controlled stochastic differential equations. 
We show that for every stochastic control system satisfying a probabilistic variant of incremental input-to- 
state stability, and for every given precision e > 0, a finite-state transition system can be constructed, which 
is £-approximately bisimilar (in the sense of moments) to the original stochastic control system. Moreover, 
we provide results relating stochastic control systems to their corresponding finite-state transition systems 
in terms of probabilistic bisimulation relations known in the literature. We demonstrate the effectiveness of 
the construction by synthesizing controllers for stochastic control systems over rich specifications expressed in 
linear temporal logic. The discussed technique enables a new, automated, correct-by-construction controller 
synthesis approach for stochastic control systems, which are common mathematical models employed in many 
safety critical systems subject to structured uncertainty and are thus relevant for cyber-physical applications. 
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1. Introduction, Literature Background, and Contributions 

The design of controllers for complex control systems with respect to general temporal specifications in a 
reliable, yet cost-effective way, is a grand challenge in cyber-physical systems research. One promising direction 
is the use of symbolic models: symbolic models are discrete and finite approximations of the continuous 
dynamics constructed in a way that controllers designed for the approximations can be refined to controllers 
for the original dynamics. The relationship between the continuity of the concrete models and the finiteness 
" ^ I of their abstractions, as well as the interplay between continuous and discrete components that is necessary 

^ to show quantitative relations between the two models, clearly categorize symbolic approaches within the 

cyber-physical domain. 

Jh When finite symbolic models exist and can be constructed, one can leverage the apparatus of finite-state 



reactive synthesis |EJ911 IMPS95[ ITho95| towards the problem of designing hybrid controllers. The formal 
notion of approximation is captured using e-approximate bisimulation relations |GP07] , which guarantee that 
each trace of the continuous system can be matched by a trace of the symbolic model up to a precision e, and 
vice versa. 

The effective construction of finite symbolic models has been studied extensively for non-probabilistic control 
systems. Examples include work on piecewise-affine and multi-affine systems with constant control distribu- 
tion |BH061 IHCS06| . abstractions based on convexity of reachable sets for sufficiently small sampling time 
[Reill] , the use of incremental input-to-state stability |GPT091 IMZHI IPGT081 IPPDTIOI |PT09j . non-uniform 
abstractions of nonlinear systems over a finite-time horizon [TI09| . and finally sound abstractions for unsta- 
ble nonlinear control systems |ZPJT12] . Together with automata-theoretic controller synthesis algorithms 
[MPS951 ITho95), effec tive symbolic models form the basis of controller synthesis tools such as Pessoa [MPT 10) 
and TuLiP |WTO+ll| . 
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However, much less is known about continuous stochastic control systems. Existing results for probabilistic 
systems include the construction of finite abstractions for continuous-time stochastic dynamical systems under 
contractivity assumptions |Aba09) , for discrete-time stochastic hybrid dynamical systems endowed with certain 
continuity and ergodicity properties |ADDll| . and for discrete-time stochastic dynamical systems complying 
with a notion of bisimulation function |AP10] . While providing finite bisimilar abstractions, all the cited 
techniques are restricted to autonomous models (i.e., with no control inputs). As such, they are of interest 
for verification purposes, but fall short towards controller synthesis goals. On the other hand, for non- 
autonomous models there exist techniques to check if an infinite abstraction is formally related to a concrete 
stochastic control system via a notion of stochastic (bi)simulation function | JP09] . however these results do 
not extend to the construction of approximations nor they deal with finite abstractions, and appear to be 
computationally tractable only in the autonomous case. Further, for specific temporal properties such as 
(probabilistic) invariance, there exist techniques to compute finite abstractions of discrete-time stochastic 
control systems [AAP+OT] . however their generalization to general properties in linear temporal logic is not 
obvious, nor their applicability to continuous-time models. The work in [Sprll] provides algorithms for the 
verication and control problems restricted to probabilistic rectangular automata, in which random behaviors 
occur only over the discrete components - this limits their application to models with continuous probability 
laws. The work in |LAB09| presents a finite Markov decision process approximation of a continuous-time 
linear stochastic control systems for the verification of given temporal properties, however the relationship 
between abstract and concrete model is not quantitative. Along the same lines, classical discretization results 
in the literature pCDOT offer approximations of stochastic control systems that are related to the concrete 
models only asymptotically, rather than according to formal bisimulation or simulation notions that are in 
the end required to ensure the correspondence of controllers for linear temporal logic specifications over model 
trajectories. 

Summing up, to the best of our knowledge there is no comprehensive work on the construction of finite 
bisimilar abstractions for continuous-time continuous-space stochastic control systems. This is unfortunate: 
these systems offer a natural modeling framework for cyber-physical systems operating in an uncertain or noisy 
environment, and automated controller synthesis methodologies can enable more reliable system development 
at lower costs and times. 

In this paper, we show the existence of e-approximate bisimilar symbolic models (in the sense of moments) for 
continuous-time stochastic control systems satisfying a probabilistic version of the incremental input-to-state 
stability property |Ang02| , for any given parameter e > 0. The symbolic models are finite if the continuous 
states lie within a bounded set. We also provide a simple way to construct the symbolic abstractions by 
quantizing the state and input sets. By guaranteeing the existence of an e-approximate bisimulation relation 
among concrete and abstract models, we show that there exists a controller enforcing a desired specifica- 
tion on the symbolic model if and only if there exists a controller enforcing an e-related specification on the 
original stochastic control system. The construction nicely generalizes results for non-probabilistic systems 
[GPTOQi IMZ12[ |PGT08| . and reduces to these results in the special case of dynamics with no noise. Further- 
more, we provide quantitative results precisely showing how the proposed e-approximate bisimulation relations 
among concrete stochastic control systems and abstract symbolic models are related to known probabilistic 
bisimulation notions recently developed in the literature. 

We finally illustrate our results on two case studies, where controllers are synthesized over (non-linear) sto- 
chastic control systems with respect to rich linear temporal logic specifications of practical relevance. 

2. Stochastic Control Systems 

2.1. Notations. The identity map on a set A is denoted by 1a- If A is a subset of B we denote hy ia ■ A ^ B 
or simply by t the natural inclusion map taking any a S ^ to z(a) = a G B. The symbols N, No, Z, M, IR+ 
and M.Q denote the set of natural, nonnegative integer, integer, real, positive, and nonnegative real numbers, 
respectively. The symbols /„, 0„, and 0„xm denote the identity matrix, the zero vector and zero matrix 
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in M"^", M", and M"^™, respectively. Given a vector x G M", we denote by Xi the i-th element of x, and 
by the infinity norm of x, namely ||a;|| = max{|a;i|, |a;2|, |x„|}, where \xi\ denotes the absolute value 
of Xi. Given a matrix M = {rriij} € M"^™, we denote by ||M|| the infinity norm of M, namely, ||M|| = 
maxi<i<„ YlT=i ^^'^ ll^lii=^ the Frobenius norm of M, namely, ||M||f = ^Tr (MM"^), where Tr(P) = 

Y^7=iPii ^'^^ ^^^y ^ ^ {Pij} ^ M"^". We denote by Ainin(^) and Aniax(^) the minimum and maximum 
eigenvalues of matrix A, respectively. The diagonal set A C M^" is defined as: A = {{x, x) \ x £ M"}. 

The closed ball centered at x e K" with radius e is defined by Be^x) = {y e E" | \\x - y\\ < e}. A set B C E" 
is called a box if B = 11^=1 "^j]' "where Ci^di G E with < di for each i e {1, . . . The span of a 

box B is defined as span{B) = min{\di — Ci\ | i = 1, . . . Define the ry-approximation [B\^ = {6 G i? | 
bi = ki7], ki d i = 1, . . . ,n} for a box B and rj < span{B). Note that [J5]^ ^ for any rj < span{B). 
Geometrically, for any ry G E+ with rj < span{B) and X > rj, the collection of sets {B\{p)}p^[B]^ is a finite 
covering of B, i.e., B C U^gj^j^ B\{p). By defining [E"],, = {a G E" | ~ kiij, ki i = 1, - ■ ■ , n}, the set 
Upg[E"]^ '^a(p) is a countable covering of E" for any 77 G E+ and A > 77. We extend the notions of span 
and approximation to finite unions of boxes as follows. Let A = IJ^l^^i' "wh^re each Aj is a box. Define 
span{A) — min {span{Aj) \ j = 1, . . . , M}, and for any 77 < span{A), define [A]ri — [j^Lii^Av 

Given a set X, a function d : X x X ^ Rq is a metric on X if for any x,y,z G X, the following three 
conditions are satisfied: i) d{x,y) = if and only if a; = y; ii) d{x,y) — d{y,x); and iii) (triangle inequality) 
d{x, z) < d{x, y)+d{y, z). Given a measurable function / : Eq — >• E", the (essential) supremum of / is denoted 
by ll/lloo; we recall that ||/||oo ■— (ess)sup{||/(t)||,t > 0}. A function / is essentially bounded if ||/||oo < 00. 
For a given time t G E"*", define fr so that fr{t) — f{t), for any t G [0,r), and frit) = elsewhere; / is 
said to be locally essentially bounded if for any r G E"*", f^. is essentially bounded. A continuous function 
7 : E^j" — > E^l", is said to belong to class JC if it is strictly increasing and 7(0) = 0; 7 is said to belong to class 
K-oD if 7 G /C and j{r) — ^ 00 as r -> 00. A continuous function /3 : Eq x EJ" — > Eg is said to belong to class 
ICC if, for each fixed s, the map j3{r,s) belongs to class /Coo with respect to r and, for each fixed nonzero r, 
the map /3(r, s) is decreasing with respect to s and /3(r, s) as s — ^ 00. We identify a relation RC Ax B 
with the map R:A^2^ defined by 6 G R{a) iff (a, b) G R. Given a relation R C A x B, R ^ denotes the 
inverse relation defined by R^^ = {{b,a) E B x A : {a,b) E R}. 

2.2. Stochastic control systems. Let (il, J-, P) be a probability space endowed with a filtration F — {J-s)s>o 
satisfying the usual conditions of completeness and right continuity |KS91|, p. 48]. Let {Ws)s>o be a p- 
dimensional F-Brownian motion. 

Definition 2.1. A stochastic control system is a tuple E = (E", U,W, /, cr), where 

• E" is the state space; 

• U C E'" is an input set; 

• hi is a subset of the set of all measurable, locally essentially bounded functions of time from intervals 
of the form [0, oo[ to U; 

• / : E"xU E" is a continuous function of its arguments satisfying the following Lipschitz assumption: 
there exist constants Lx,Lu G E+ such that: \\f{x^u) — f[x' ,u')\\ < Lx\\x — a;'|| + iM||7i — u'\\ for all 
X, x' G E" and all u, u' G U; 

• a : E" — >■ E"^P is a continuous function satisfying the following Lipschitz assumption: there exists a 
constant Z G E+ such that: ||cr(a::) — f(a;')|| < Z\\x — x'\\ for all x,x' G E". 

A stochastic process ^ : x [0, oo[— ?> E" is said to be a solution process of S if there exists v GU satisfying: 
(2.1) d^^fi^,v)dt + aiOdWu 

P-almost surely (P-a.s.), where / is known as the drift, a as the diffusion, and again Wt is Brownian motion. 
We also write ^au(i) to denote the value of the solution process at time t G under the input curve v 
from initial condition ^at;(0) — a P-a.s., in which a is a random variable that is measurable in Tq. Let us 
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remark that J-q in general is not a trivial sigma-algebra, and thus the stochastic control system E can start 
from a random initial condition. Let us emphasize that the solution process is uniquely determined, since the 
assumptions on / and a ensure its existence and uniqueness |Oks02[ Theorem 5.2.1, p. 68]. 



3. A Notion of Incremental Stability 



This section introduces a stability notion for stochastic control systems, which generalizes the notion of 
incremental input-to-state stability (5-ISS) |Ang02| for non-probabilistic control systems. The main results 
presented in this work rely on the stability assumption discussed in this section. 

Definition 3.1. A stochastic control system E — (M", \J^U,f,a) is incrementally input-to-state stable in the 
qth moment (5-ISS-Mq), where q > 1, if there exist a ICC function j3 and a /Coo function 7 such that for any 
t G , any MJ^ -valued random variables a and a' that are measurable in J-'q, and any v, v' G lA , the following 
condition is satisfied: 

E Ma.it) - ia'.'{tW] < /3 (E [||a - a'lr] , + 7 (11^ - v'\U . 



(3.1) 



It can be easily checked that a 5-ISS-Mg stochastic control system E is (5-ISS in the absence of any noise as in 
the following: 

(3.2) ||U(<) - ^a'.'im <P{\\a- a'W , i) + 7 {\\v - v'\\^) , 

for a, a' G M", some j3 e /C£, and some 7 G /Coo- Moreover, whenever /(0„,0m) = 0„ and (t(0„) = 0„xp (i-e., 
the drift and diffusion terms vanish at the origin), then (5-ISS-Mg implies input-to-state stability in the qt\\ 
moment (ISS-M^) |HM09j and global asymptotic stability in the qth moment (GAS-M,) |CL06j . 

Similar to the use of J-ISS Lyapunov functions in the non-probabilistic case |Ang02| , we now describe (5-ISS-Mq 
in terms of the existence of incremental Lyapunov functions, as defined next. 

Definition 3.2. Consider a stochastic control system E — (M", U,Z^, /, cr) and a continuous function V : 
R" X M" — M.Q that is smooth on {M" x ]R"}\A. The function V is called an incremental input-to-state stable 
in the qth moment (5-ISS-Mq) Lyapunov function for E, where q > I, if there exist /Coo functions a, a, p, 
and a constant k G , such that 

(i) a (resp. a) is a convex (resp. concave) function; 

(ii) for any x, x' G M", 



a{\\x-x'\\i) < V{x,x') <a\ 



\x — X 



(iii) for any x, x' G M", x 7^ x' , and for any u, u' G U, 



V{x,x') := [d.,V d^,V] 



f{x,u) 
f{x',u') 



Tr 



a{x) 
<j{x') 



K(x) a^ix')] 



dx'.xV dx'.x'V 



< -kV{x,x')+p{\\u-u'\\), 



where is the infinitesimal generator associated to the stochastic control system (2.1 1 jOks02| Section 

7.3], which in this case depends on two separate controls u,u' . The symbols dx and dx,x' denote first- and 
second-order partial derivatives with respect to x and x' , respectively. 



Roughly speaking, condition {ii) implies that the growth rate of functions a and a are linear, as a concave 
function is supposed to dominate a convex one. However, these conditions do not restrict the behavior of 
a and a to only linear functions on a compact subset of M". Note that the condition (i) is not required in 
the context of non-probabilistic control systems. The following theorem clarifies why such a requirement is 
instead necessary for a stochastic control system, and describes 5-ISS-Mg in terms of the existence of J-ISS-M, 
Lyapunov functions. 

Theorem 3.3. A stochastic control system E = (M", U, W, /, a) is 5-ISS-Mq if it admits a S-ISS-Mq Lyapunov 
function. 
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Proof. The proof is a consequence of the apphcation of GronwaU's inequahty and of Ito's lemma [Oks021 pp. 
80 and 123]. Assume that there exists a 5-ISS-Mg Lyapunov function in the sense of Definition 3.2 For any 
t € any v, v' & U, and any IR"-valued random variables a and a' that are measurable in J-q, we obtain 

< E 

/ TS[ViU{s),^a'v'{s))]ds + nV{a,a')] + p{\\v ~ v'\\oo% 
Ja 

which, by virtue of GronwaU's inequality, leads to 

(3.3) E [V{U,{t),^a'v'm < E[V{a, a')]e'«* + te'-'p{\\v ~ < E[F(a, a')]e-' + —p{\\v - v'\\^). 



V{a,a')+ / {~KV{U{s),^a'v'{s)) + p{\\v{s) - v'{s)\\))ds 



Hence, using property (ii) in Definition 3.2 we have 

a (E W - L'v'itW]) < E [a (||U W - Ca'.' WD] < E W, Ca'.' W)] 



(3.4) 



< nVia, a')]e-«* + -p{\\v - v'\U < E [a (|ja - a'||«)] e"''* + -p{\\v - v'\U 

< a (E - a'W"]) e^^' + —p{\\v - u'||oo), 



where the first and last inequalities follow from property (i) and Jensen's inequality |Oks02| p. 310]. Since 
a € /Coo, the inequality (3.4 1 yields 



E [||U(t) - ia'.'itW] < a-' (E [|la - a'|l^]) e"-* + ^p{\\v - v'\U 

< a-^ {a (E [\\a - a']]"]) e""^* + a (E [||a - a'||«]) e""*) 



— p{\\v - v'Woo) + —P{\\v - w'lloo) 

en eK 

< (2a (E [\\a - a'^]) e""*) + a'^ " ^'ll-)) 



Therefore, by introducing functions /3 and 7 as 



/? (E [\\a - a'||«] , t) (2a (E [||a - a'|l']) e""*) , 7 (Ik " ^'lU) (|:P(lk - ^'lU)) , 



condition (3.11 is satisfied. Hence, the system S is J-ISS-M^ 



□ 



One can resort to available software tools, such as SOSTOOLS |PPSP04] . to search for appropriate, non-trivial 
i5-ISS-Mq Lyapunov functions for system S. 

Now we look into special instances where function V can be easily computed based on the model dynamics. 
The first result provides a sufficient condition for a particular function to be a (5-ISS-Mq Lyapunov function 
for a stochastic control system E, when g = 1, 2 (that is, in the first or second moment). 

Lemma 3.4. Consider a stochastic control system S = (M", U, /, cr). Let q e {1,2}, P E M"^"- be a 
symmetric positive definite matrix, and the function V : M" x M" — >■ be defined as follows: 

(3.5) V{x, x') := (V{x, x')) ' = Q (a; - x'f P {x - x') 
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and satisfy 
(3.6) 

or, if f is dijferentiable with respect to x, satisfy 



{x-x'fP{f{x,u)-f{x\u)) + \ ^ {a{x) - a{x')) ' < -Ti {V {x , x' )Y 



(3.7) 



[x-x'Y Pd.J{z,u){x~x') + 



P {a{x) - a{x')) < -k{V{x,x'))^ , 

F 



for all X, x' , z G M", for all it G U, and for some constant k G Then V is a d-ISS-Mg Lyapunov function 
for E. 



The proof of Lemma |3.4| is provided in the Appendix. 



The next result provides a condition that is equivalent to (3.6) or (3.7) for linear stochastic control systems 



S (that is, for systems with linear drift and diffusion terms) in the form of a linear matrix inequality (LMI), 
which can be easily solved numerically. 

Corollary 3.5. Consider a stochastic control system E = (M", U,L{,f,a), where for all x G M", and u € U, 

f{x,u) := Ax + Bu, for some A G M"^", B G M"^", and a{x) := [aix (J2X ■■■ a^x], for some en G M"""". 
Then, function V in (3.5 1 is a S-ISS-Mq Lyapunov function for S, when q G {1,2}, if there exists a constant 
K G satisfying the following LMT. 



(3.8) 



PA + A^P + ^ crJPa, < -kP. 



Proof. The corollary is a particular case of Lemma [3. 4| It suffices to show that for linear dynamics, the LMI 



in (3.8) yields the condition in (3.6) or (3.7). First it is straightforward to observe that 

p 



Vp i<7{x) ~ <y{x'))j^ = Tr ((a(x) - aix')f P {a{x) - a{x'))'^ = (x - x'f ^ afPa,{x - x'). 



and that 



{x - x'fPd,f{z,u){x -x') = -ix- x'f {PA + A^P) {x - x'), 



for any x, x', z G M" and any u G U. Now suppose there exists k G M+ such that (3.8) holds. It can be readily 



verified that the desired requirements in (3.6) and (3.7) are verified by choosing k 



~2- 



□ 



As a practical consequence of the previous corollary, in order to obtain tighter upper bounds in (3.1 ) one can 
seek appropriate matrices P by solving the LMI in (3.8). 

Remark 3.6. Consider a stochastic control system Y, — (M", \J,U, f,a). Assume that f is differentiable with 
respect to x and that, for any x G M", o'(x) :~ [aix (72X ■ ■ ■ (Jpx], for some ai G M"^". Then, the function V 
in (3.5 ) is a S-ISS-Mq Lyapunov function for E, when q G {1, 2}, if there exists a constant k G M+ satisfying 
the following matrix inequality: 



(3.9) 



p 

Pd,f{z, u) + (d^fiz, u)f P + Y. aJPa, < 

i=l 



HP, 



for any z G M" and any u G U. One can easily verify that condition [3.9) corresponds to the contractiv- 
ity conditions (with respect to the states) in |LS98[ IZTllj . obtained with contraction metric P, and to the 
Demidovich's condition in |PvdWN05| for a system E in the absence of any noise, i.e. ai — Onxn for all 
i = l,---,p. □ 
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3.1. Noisy and noise-free trajectories. In order to introduce a symbolic model in Section [5] for tire sto- 
chastic control system, we need the following technical results, which provide an upper bound on the distance 
(in the qth moment) between the solution processes of E and those of the corresponding non-probabilistic 
control system obtained by disregarding the diffusion term (that is, a). 

Lemma 3.7. Consider a stochastic control system E — (M", U,Z^, /, cr) such that /(0„,0m) = 0„, and 
c(On) — 0„xp- Suppose there exists a S-ISS-Mq Lyapunov function V for E such that its Hessian is a positive 
semidefinite matrix in K2nx2n q > 2. Then for any x in a compact set D and any v ^ U, we have 



(3.10) 



E 



where is the solution of the ordinary differential equation (ODE) ^xi,{t) = f {£,xv{^):''-'{^)) starting from 
the initial condition x, and the nonnegative valued function h tends to zero as t ^ 0, f — > +oo, or as Z ^ 0, 
where Z is the Lipschitz constant, introduced in Definition\2.1\ 



The proof of Lcmma|3.7|is provided in the Appendix. One can compute explicitly the function h using equation 



(9.4) in the Appendix. 



Remark 3.8. In the previous lemma, if V is a polynomial, then the conditioi^H(y){x,x') >z 02nx2n for all 
x,x' e M" is equivalent to V being a convex function [ CH08, . Furthermore, if we assume that for allx,x' € M", 
H{V){x, x') = M{x,x')'^M(x,x'), where M{x,x') € R"^^" is a polynomial matrix for some s E N, then V 
is a sum of squares which can be efficiently searched through convex linear matrix inequalities optimizations 
[CH08| . and using some available tools, such as SOSTOOLS IPPSP04] . □ 



The following lemma provides an explicit result in line with that of Lemma |3.7| for a model E admitting a 
i5-ISS-Mq Lyapunov function V as in (3.5 1, where q G {1,2}. 

Lemma 3.9. Consider a stochastic control system E (M", \J,U, f, a) such that /(0„, Om) — 0„, and (t(0„) = 
Onxp- Suppose that the function V in (3.5) satisfies (3.6) or (3.1) for E. For any x in a compact set D and 

any V G U , we have Wj \\S,xv{t) — £,xv{^)\ 

2 ^/P 

h{a,t) = 



r? vani{n,p\Z^& 



< h{a, t), where 

V 



sup {\\xf} 



- sup < 
uev 



2.1 



and is the solution of the ODE ^^vi^) 



Z is the Lipschitz constant, as introduced in Definition 

/ (^^^^(t), u(t)) starting from the initial condition x. It can be readily verified that the nonnegative valued 
function h tends to zero as t 0, i — > +oo, or as Z — > 0. 

The proof of Lemma |3.9| is provided in the Appendix. 



For a linear stochastic control system E, the following corollary tailors the result in Lemma 3.9 obtaining 
possibly a less conservative expression for function h, based on parameters of drift and diffusion. 

Corollary 3.10. Consider a stochastic control system E = (M", \J,U, f,a), where for all x G K" and u G M™, 
f{x) := Ax + Bu, for some A GjR"^", J5 e IT^", and a{x) [aix a2X ■■■ cipx], for some ai e R"^". 



Suppose that the function V in (3.51 satisfies [3.8) for E. For any x in a compact set D and any v GlA, we 

r 1 1 ~ 1 1 21 

/laue E I Kxu (i) — ^ajjj (0 < h{a,t), where 



h{a, t) 



\i=l 



sup {11x11} + 



/ ||e'^''S||dr) sup{||u||}) ds, 



(^) 

where is the solution of the ODE ^^vi^) — -^^xvi^) + Bv{t) .starting from the initial condition x. 



^The notation H(V){x, x') denotes the value of Hessian matrix of V at (x, x') S ' 
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The proof of Corollary |3.10| is provided in the Appendix. 



4. Symbolic Models 

4.1. Systems. We employ the notion of system |Tab09| to describe both stochastic control systems as well 
as their symbolic models. 

Definition 4.1. A system S is a tuple S — {X,Xo,U, — >,Y,H), where X is a set of states, Xq Q X is a 
set of initial states, U is a set of inputs, — >-C X y. U y. X is a transition relation, Y is a set of outputs, and 
H : X Y is an output map. 

A transition {x,u,x') G — > is also denoted by x — x'. For a transition x — x' , state x' is called a 
It-successor, or simply a successor, of state x. For technical reasons, we assume that for any x € X and u € U, 
there exists some u-successor of x — let us remark that this is always the case for the considered systems later 
in this paper. 

System S is said to be 

• metric, if the output set Y is equipped with a metric d : F x F — > Mj; 

• finite, if X is a finite set; 

• deterministic, if for any state x G X and any input u, there exists exactly one u-successor. 

For a system S — {X, Xq, U, — >, Y, H) and given any state xq £ Xq, a finite state run generated from xq is a 

finite sequence of transitions: xq — xi — ^ X2 — ^ ■ ■ ■ ^ a^n-i >- Xn, such that Xi — ^ a^i+i for 
all < * < 'T-- A finite state run can be directly extended to an infinite state run as well. 

4.2. System relations. We recall the notion of approximate (bi)simulation relation, introduced in |GP07) . 
which is useful when analyzing or synthesizing controllers for deterministic systems. 

Definition 4.2. Let Sa = {Xa,Xao,Ua, ,Ya,Ha) and Sb ^ {Xb,XbQ,Ub, — ^ ,Yb,Hb) be metric sys- 

tems with the same output sets Ya — Yb and metric d. For e G M''", a relation R C Xa x Xb is said to be an 
e-approximate simulation relation from Sa to Sb if the following three conditions are satisfied: 

(i) for every Xao G Xao, there exists Xbo € Xbo with {xao,Xbo) G R; 

(ii) for every {xa,Xb) G R we have A{Ha{xa), Hb{xb)) < e; 

(iii) for every {xa,Xb) G R we have that Xa "° ► x'^ in Sa implies the existence of Xb "'' ► x'^^ in Sb satis- 
fying e i?. 

A relation R C Xa x Xb is said to be an e-approximate bisimulation relation between Sa and Sb if R is an 
e-approximate simulation relation from Sa to Sb and R^^ is an e-approximate simulation relation from Sb to 

Sa- 

System Sa is e- approximately simulated by Sb, or Sb e -approximately simulates Sa, denoted by Sa ^5 Sb, if 
there exists an e-approximate simulation relation from Sa to Sb- System Sa is e-approximate bisimilar to Sb, 
denoted by Sa ^5 Sb, if there exists an e-approximate bisimulation relation R between Sa and Sb- 

Note that when e = 0, condition (ii) in the above definition is modified as {xa,Xb) G i? if and only if 
Ha{xa) — Hb{xb), and R becomes an exact simulation relation, as introduced in |Mil89j . Similarly, whenever 
e ~ 0, R possibly becomes an exact bisimulation relation. 
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5. Symbolic Models for Stochastic Control Systems 



This section contains the main contribution of the paper. We show that for any stochastic control system S ad- 
mitting a 5-ISS-Mg Lyapunov function, and for any precision level e G we can construct a finite system that 
is e-approximately bisimilar to E. In order to do so, we use the notion of system as an abstract representation 
of a stochastic control system, capturing all the information contained in it. More precisely, given a stochastic 

control system E = {W^ ,U,U , f,a), we define an associated metric system S{T,) — {X,Xo,U, ,Y,H), 

where: 

• X is the set of all M"-valued random variables defined on the probability space {ft,J-, P); 

• Xq is the set of all IR"-valucd random variables that are measurable over the trivial sigma- algebra Tq, 
i.e., the system starts from a deterministic initial condition, which is equivalently a random variable 
with a Dirac probability distribution; 

• C/=W; 

• X — ^ x' if X and x' are measurable in J^t and J^t+r, respectively, for some t g and r G and 
there exists a solution process ^ : x Rq — > R" of S satisfying ^{t) = x and £,xv{t) = x' P-a.s.; 

• y is the set of all M"-valued random variables defined on the probability space {il,J-,T); 

We assume that the output set Y is equipped with the natural metric d{y, y') = (E — ' , for any 

y,y' G Y and some q > I. Let us remark that the set of states of S{T,) is uncountable and that S'(S]) is 



a deterministic system in the sense of Definition 4.1 since (cf. Subsection |2.2[ ) the solution process of E is 
uniquely determined. 

The results in this section rely on additional assumptions on model E that are described next (however such 
assumptions are not required for the definitions and results in Sections [2] |3j and |4]) . We restrict our attention 
to stochastic control systems E = (M", U,U,f,a) with /(0„,0m) = 0„, (t(0„) — Onxp, and input sets U that 
are assumed to be finite unions of boxes. We further restrict our attention to sampled-data stochastic control 
systems, where input curves belong to set Ur which contains only curves that are constant over intervals of 
length r G M+, i.e. 

llr = [v:R+ \ v{t) = v{{k - 1)t), t G [{k - l)r, fcr[, fc G n|. 



Let us denote by S'r(E) a sub-system of S'(E) obtained by selecting those transitions from 5(E) corresponding 
to solution processes of duration t and to control inputs in lAr- This can be seen as the time discretization or 
as the sampling of a process. More precisely, given a stochastic control system E = (M", U,Z^r, /, f), we define 

the associated metric system 5,-(E) — (^Xr,Xro,Ur, *- ,Yr,Hr^, where Xr = X, Xro = Xq, Ur = Ur, 

Yr = Y, Hr^ H, and 

• Xr >- x'r if Xr and x'^ are measurable, respectively, in J^kr and J^(fc+i)T for some k G Nq, and there 
exists a solution process ^ : x — ^ M" of E satisfying ^(fcr) ~ Xr and £,x^v^{t) = x'r P-a.s.. 

Notice that a finite state run xq "° >- xi — ^ ... " >- xn, of S't(E), where Vi G Ur and Xi — C^i-i-ui-i (''") for 
i — 1, - ■ ■ ,N, captures the solution process of the stochastic control system E at times t = 0,t, ■ ■ ■ , Nt, started 
from the deterministic initial condition xq and resulting from a control input v obtained by the concatenation 
of the input curves Vi (i.e. v{t) = Vi-i{t) for any i G [(i — l)r, it[), for « = 1, • • • , A^. 

Let us proceed introducing a fully symbolic system for the concrete model E. Consider a stochastic control 
system E = (M", U, Wr, /, u) and a triple q — {T,rj^^) of quantization parameters, where r is the sampling 
time, 77 is the state space quantization, and ^ is the input set quantization. Given E and q, consider the 
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following system: 
(5.1) 



(Xq, XqO, ?7q, 



consisting of (cf. Notation in Subsection 2.1): 



qo = 



if there exists a Xq G such that 



(t) - < 77, where ^^_^„_^(t) = / (t), UqW) ; 



• Yq is the set of all ]R"-valued random variables defined on the probability space (fi, J^, P); 



X„ 



Y„ 



Note that we have abused notation by identifying G [U]^ with the constant input curve with domain [0, r[ 
and value Wq. Notice that the proposed abstraction S'q(I]) is a deterministic system in the sense of Definition 
In order to establish an approximate bisimulation relation, the output set Y^ is defined similarly to the 



4.1 



stochastic system SrC^)- Therefore, in the definition of _ffq, the inclusion map i is meant, with a slight abuse 
of notation, as a mapping from a deterministic grid point to a random variable with a Dirac probability 
distribution centered at the grid point. As argued in |Tab09j . there is no loss of generality to alternatively 
assume that Yq = Xq and iJq = Ix, . For later use, we denote by "^x^on. ■ ^ X^^ a state run of 5'q(E) from 
initial condition X2;^pu(0) = Xqo and under input sequence u : No — > [U]^. 

The transition relation of 5q(S) is well defined in the sense that for every Xq G P"]r) and every G [U]^ 
there always exists x'^ G [M"]^ such that 
X G M" there always exists a state x' G [1 



a state x'^ G 



V satisfying ^ (t) - x' 



— ^ Xq. This can be seen since by definition of [M"]^, for any 
such that \\x — x'\\ < -q. Hence, for „ (r) there always exists 



< 



In order to show the first main result of this work, we raise a supplementary assumption on the (5-ISS-Mq 
Lyapunov function V as follows: 

(5.2) \V{x,v) -V{x,z)\ <%\\v - z\\), 

for any x,y,z G M", and some A^oo and concave function 7. This assumption is not restrictive, provided V is 
restricted to a compact subset of M" x M". Indeed, for all x,y,z ^ D, where D C M" is compact, by applying 
the mean value theorem to the function y — > V{x,y), one gets 

dV{x,y) 



\V{x, y) - V{x, z)\ < 7 (||y - zll) , where 7(r) 



max 

x,yeD\A 



dy 



In particular, for the (5-ISS-Mi Lyapunov function V defined in (3.51, we obtain explicitly that j{r) = 
d£}=r |Tab09[ Proposition 10.5]. We can now present the first main result of the paper, which relates the 



existence of a J-ISS-M^ Lyapunov function to the construction of a symbolic model. 

Theorem 5.1. Consider a stochastic control system S ~ (M", U,Z^t, /, c), admitting a 6-ISS-Mq Lyapunov 
function V, of the form of {3.5) or the one explained in Lemmas 3.7 For any e G and any triple 
q — {T,ri,fj,) of quantization parameters satisfying /i < span{\J) and 

(5.3) 

(5.4) e- 
we have that S'q(E) =5 Sr{^). 



(e-?) + Ip(^) + 7 ({h{a, r)) ^ + 77) < a (£«) , 



It can be readily seen that when we are interested in the dynamics of S, initialized on a compact D C M" of 
the form of finite union of boxes and for a given precision e, there always exists a sufficiently large value of t 
and small enough values of 77 and /i, such that 77 < span(D) and such that the conditions in (5.3) and (5.4) 
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are satisfied. On the other hand, for a given fixed sampling time r, one can find sufficiently small values of rj 
and /i satisfying 77 < span{D), (5.3) and (5.4), as long as the precision e is lower bounded by: 

'7 ((/i(cr,r))' 

(5-5) e>\a'\ \ 



One can easily verify that the lower bound on e in (5.5) goes to zero as r goes to infinity or as Z — > 0, where Z 
is the Lipschitz constant, introduced in Definition |2.1[ Furthermore, one can try to minimize the lower bound 
on £ in (5.5 1 by appropriately choosing a (5-ISS-Mq Lyapunov function V (cf. Section [s]). 

Proof. We start by proving that S'r (S) ^5 "^q (S) . Consider the relation R C x Xq defined by {xr , Xq ) G i? if 
and only if E[V {Hr{xr), H^ix^))] =E[V {xr,Xq)] < «(£'?). Since Xro C Uj,e[R"]„ I3jj{p), for every Xro e Xro 
there always exists Xqo G X^q such that ||xro — a;qo|| < rj. Then, 

E [V{xra,Xqo)] = V{xro, x^q) < adl^^o - a^qoll'') < a ('7'^) < (e'^) i 



because of (5.3 1 and since a is a /Coo function. Hence, {xT-o,Xqo) e R and condition (i) in Definition 
satisfied. Now consider any {xr,Xq) E R. Condition (ii) in Definition 4.2 is satisfied because 



4.2 



(5.6) 



(E[||a;, -Xq||«])^ < {a-^{E[V{xr,Xq)\)Y < e. 



We used the convexity assumption of a and the Jensen inequality |Oks02j to show the inequalities in (5.6 1. 
Let us now show that condition (iii) in Definition 4.2 holds. Consider any Vr eUt- Choose an input Mq G J7q 

satisfying 



(5.7) 



\\Vt - Uqlloo = ||Wr(0) - Wq(0)|l < ^. 



Note that the existence of such Wq is guaranteed by U being a finite union of boxes and by the inequality 
fjL < span{U) which guarantees that U C IJpgju] ^ii{p)- Consider the transition Xr * x'^ = i^^^iv (''') in 
5t(S). Since F is a 5-ISS-Mg Lyapunov function for S, in light of (3.3) as well as (5.7), we have 

(5.8) E [y(4, (r))] < E [V{xr,Xq)] e-«^ + —p{\\vr - Uq||oo) < a (e«) e"''" + —pUi). 

Since M" C IJ grjj,ii Br]{p), there exists a. x' E X^ such that 



(5.9) 



Ca;qUq(''") -^q 



which, by the definition of S'q(E), implies the existence of S'q(E). Using Lemmas 



3.7 



3.9 



the 



concavity of 7, the Jensen inequality |Oks02) . the inequalities (5.2), (5.4), (5.8), (5.9), and triangle inequality, 
we obtain 

E [V{x',,x'q)] = E [T/(x;,e.,„,(r)) + Vix'^,x'^) - ^(4, e..^, (r))] 

= E [V{x'^,^,^uSr))] +E [y(4,x;) ~ F«,C.qn,(T))] 

< a (e«) e-«- + ^p{fi) + E [7 (||ex,n,(r) - x^] 

< a (e«) e-«- + ^p{p) + 7 (e [ C.^n, (r) - L,u, (r) + L,u, (r) ^ ]) 

(£')e-^^ + ^p(M)+7(E[ e.,n,(r)-L,„,(T) 1+ L^u^r) ~ x'^ ) 
1 



< a 



<a(e«)e-'^" + — p(/i)+7 (/i(ct,t))^ +r; <«(£-?). 
en \ / 

Therefore, we conclude that (^x'^,x'q) E R and that condition (iii) in Definition 



4.2 



holds. 
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Now we prove 5'q(S) St-{^) implying that is a suitable e-approximate simulation relation from S'q(S) 
to S't-(S). Consider the relation R C Xr x X^, defined in the first part of the proof. For every Xqo G ^qo, by 
choosing Xtq = a^qo, we have V {xtq, Xqo) = and (x^-q, x^q) € R and condition (i) in Definition 
Now consider any {xr,Xq) E R. Condition (ii) in Definition 4.2 is satisfied because 



4.2 



is satisfied. 



(5.10) 



(E[||x,-xq||«])^ < (a-i(E[l/(x,,Xq)]))' <e. 



We used the convexity assumption of a and the Jensen inequality |Oks02j to show the inequalities in (5.101. 
Let us now show that condition (iii) in Definition 4.2 holds. Consider any Uq € Uq. Choose the input = Uq 



and consider x'^ = ^^^^^.^(t) in St{T,). Since is a J-ISS-M^ Lyapunov function for S, one obtains: 



(5.11) 



E[l/(x;,ex,u,(T))] <e-'^-nVixr,Xq)]<e-^-a{e^). 



Using Lemmas 3.7 or 3.9 the definition of S'q(S), the concavity of 7, the Jensen inequality |Oks02] . the 
inequalities (5.2 1, ( |5.4| ), ( |5.1l] ), and the triangle inequality, we obtain 

E [V (4, x;)] ^E[V «, (r)) + F (4, x'^) - y (4, (r))] 

= E [l^(4,^.,„,(r))] +E[V{x',,x'q) - T/(4,e.,„,(r))] 

<e--^a(e'^)+E[7(||e.,„,(r)-x'q||)] 

< e-^^a (e^) + 7 (e (r) - (r) + (r) - x'^ 

< e-«-a (£^) + 7 (e [||^.,„,(r) - 4,«,(r)||] + ||?.,„,(t) - x'^ 

< e-'=^a(e9) +7((/i((7,t))^ +r/) < a{e'') . 
Therefore, we conclude that (x'^,Xq) € R and condition (iii) in Definition 



4.2 



holds. 



□ 



Note that the results in |GPT09j . as in the following corollary, are fully recovered by the statement in Theorem 
5.1 if the stochastic control system E is not affected by any noise, implying that h{a,T) is identically zero and 
that the (5-ISS-Mq property reduces to the (5-ISS property. 

Corollary 5.2. Let S = (M", \J,Ur, f,Onxp) be a 6-ISS control system admitting a 6-ISS Lyapunov function 
V . For any e G and any triple q — (r, ry, [i) of quantization parameters satisfying /j < span(U) and 

(5.12) a (^) <«(£), 

1 



(5.13) 

one obtains Sq(Ti) 



The next main theorem provides a result that is similar to that in Theorem |5.1[ which is however not obtained 
by explicit use of (5-ISS-M^ Lyapunov functions, but by using functions /? and 7 as in (3.1 1. 



Theorem 5.3. Consider a 5-ISS-Mq stochastic control system E = {M." ,\J,Ut, f,cr), satisfying (3.10). For 
any e G M^ , and any triple q = (r, rj, fi) of quantization parameters satisfying fi < span{\J) and 

(5.14) 

we have S'q(S) =g SrC^). 



{(i {e\ t) + 7(Ai))^ + (/i(a,T))^ + r/ < e, 



It can be readily seen that when we are interested in the dynamics of E, initialized on a compact D C M" 
of the form of finite union of boxes and for a given precision e, there always exists a sufficiently large value 



of r and small values of rj and /x such that 77 < span{D) and the condition in (5.14) are satisfied. However, 



unlike the result in Theorem |5.11 notice that here for a given fixed sampling time r, one may not find any 

1' '1 1 

values of 77 and /x satisfying (5.14) because the quantity (/3 (e'^,r))'' may be larger than e. As long as there 

exists a triple q, satisfying (5.14), the lower bound of precision e can be computed by solving the following 
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inequality with respect to e: £ — (/3(e*,r))' > (/i(o', t)) « . In this case, one can easily verify that the lower 
bound on e goes to zero as r goes to infinity, as t goes to zero (only if {(3 (e*, 0)) « < e), or as Z 0, where 
Z is the Lipschitz constant introduced in Definition 2.1 The symbolic model S'q(S), computed by using the 



quantization parameters q provided in Theorem 5.3 whenever existing, is likely to have fewer states than the 
model computed by using the quantization parameters provided in Theorem |5.H or may provide a better lower 
bound on e for a fixed sampling time r. We refer the interested reader to the case studies section for a detailed 
comparison between the results of Theorems |5 . 1| and |5 . 3| on some practical examples. 



Proof. We start by proving '5'q(S). Consider the relation R C Xt- x defined by (a;^, Xq) G i? if and 



only if (E[||iJ,(xO-ifq(xq)||^])' = (E - Xqll"]) ' < s. Since C Up^p 
there always exists Xqo G X^q such that \\xt-o — Xqo\\ < rj. Then, 

(E [||a;,o - a^qoll"])' = (l^ro - XqoD' < < £, 



B.,j{p), for every Xro e 



because of (5.14). Hence, (a^rOi^^qo) G R ^iid condition (i) in Definition 4.2 is satisfied. Now consider any 
(xt, Xq) S R. Condition (ii) in Definition 4.2 is satisfied by the definition of R. Let us now show that condition 
(iii) in Definition 



4.2 



(5.15) 



holds. Consider any Vt G Ur- Choose an input Uq G f/q satisfying 

\\Vt - Mqlloo = II^^t(O) - Uq(0)|| < ^I. 



Note that the existence of such Uq is guaranteed by U being a finite union of boxes and by the inequality 
fj, < span{[J) which guarantees that U C Upgjy] B^{p). Consider the transition Xr — ^ x'^ = £,XrVr{''') in 
5t(S). It follows from the (5-ISS-Mg assumption on S and (5.15) that: 

(5.16) E - C.,«,(T)f ] < /? (E [\\xr - x,\n , r) + j{\\vr - UqlU) < P {e", r) + 

Since M" C lJpg[R>>]^ Brf{p), there exists x'^ G X^ such that 



(5.17) 



^x„u„i''') "^q 



< 



which, by the definition of S'q(S), implies the existence of Xq 
(5.14), (5.16), (5.17), and triangle inequality, we obtain 



in S'q(S) 



Using Lemmas 3.7 or 3.9 



0, 



(E[||4 



] ) ' = (e x',- (r) + (r) - e,^„^ (r) + (r) - x'^ 



< (E - e.,.,(r)|r])' + (e [||C.,„,(t) - (x^uSr)f]y + (e [||e,,„,(T) - x^ll']) 



< (/3(e«,r)+7(/i))^ + (/^(a,T))5 +r;<e. 



4.2 



holds. 



Therefore, we conclude that (a;^,a;^) G R and that condition (iii) in Definition 

Now we prove >5'q(S) S't(S) implying that is a suitable e-approximate simulation relation from S'q(S) 
to S't(S). Consider the relation R C Xr x X^, defined in the first part of the proof. For every a;qo G ^q o, 
by choosing Xro = x^q, we have 



\Xt-Q — a^qol 



and {xro,Xqo) G R and condition (i) in Definition 



4.2 



IS 



satisfied. Now consider any {xr,Xq) G R. Condition (ii) in Definition 4.2 is satisfied by the definition of R. 
Let us now show that condition (iii) in Definition 4.2 holds. Consider any Uq € Uq. Choose the input Vr = Uq 
and consider a;^ = ^x^v-ri''') in '5't(S). Since E is (5-ISS-Mq, one obtains: 



(5.18) 



E [\\x'r - e.,«,(T)f ] < /3 mi^r - xqinr) < /3 (e", r) . 
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Using Lemmas 13. 71 or 3.9 the definition of S'q(I]), (5.141, (5.18), and the triangle inequality, we obtain 



{^[\Wr-<r]y 



E 



£.x,u, (t) + (,^^u, (r) - Cx„u„ (t) + Cx„u„ (t) - x' 



E 



< (E[||<-^,,„,(r)f])' 

<(/3(e«,r))^+(/^(a,T))^+r;<e. 
Therefore, we conclude that {x'^,x'^) € R and condition (iii) in Definition 



E 



4.2 



holds. 



□ 



Note that the resuhs in jPGTOSj . as in the following corollary, are fully recovered by the results in Theorem 



5.3 if the stochastic control system E is not affected by any noise, implying that h(a,T) is identically zero and 
that the (J-ISS-M^ property becomes the (5-ISS property. 



Corollary 5.4. Let E — (M", U,Ur, /, 0„xp) be a 5-ISS control system (i.e. satisfying (3.2) ). For any e G M"*", 
and any triple q — {r^rj,^) of quantization parameters satisfying /i < span{\J) and 

(5.19) /3(e,T)+7(M) + 77<e, 

we have S'q(E) =^ S',-(E). 



Symbolic models can be easily model checked or employed towards controller synthesis. It is of interest to 
understand how abstract controllers can be used over the concrete models. The next proposition elucidates 
how a controller Scant synthesized to solve a simulation game over S'q(S) can be refined to a controller for 
S't-(E). a detailed description of the feedback composition (denoted by ||) and of its properties for metric 
systems can be found in |Tab09j . 

Proposition 5.5. jTab09[ Proposition 11.10] Consider a stochastic control system S, and a specification 
described by a deterministic system S, 

subset O/M", XspecQ Q Xspec, U^pec = 



spec — ^ 
spec\ 7 



spec 



Y H 

I ^ spec 1 spec 



where X 



spec 



^ Xspec ^ U spec ^ Xgpcc: ^spec the SCt of all 



spec "is a finite 
valued 



random variables defined on the probability space (f2,J^, P), and H, 



spec 



^ ■ Xgpcc ^ ^spec • 



issume 



that 



SrC^) =5 '5'q(S) and Scant is synthcsizcd to solve exactly a simulation game for iS'q(E) and a specification 
'S'q(E) ^5 Sspec (resp. Scant \\ Sc^{T.) Sspec)- Then, using S'cont = Scant II S'q(i;) as a 



Sspec ■ Scant 



controller for Sr{^), we obtain: S'^^ 



Srm ^ 



Q Sspec 



(resp. S'cont II '5'r(S) 



Remark 5.6. Although we assume that the set U is infinite, Theorems 5.1 (resp. Corollary \5.^ and 5.3 
(resp. Corollary 5.4) still hold when the set U is finite, with the following modifications. First, the system E 
is required to satisfy the property (3.1) for v = v' . Second, take — \J in the definition of SqCE). Finally, in 
the conditions (5.4-) (resp. (5.13lj and (5.14) (resp. (5.19)j set ^ — 0. □ 



Remark 5.7. // stochastic control system E is not 5-ISS-M^ 



1' 



one can use 



the results in [ZEAL 12] . providing 
symbolic models that are only sufficient in the sense that the refinement of any controller synthesized for the 
symbolic model enforces the same desired specifications on the original stochastic control system (in the sense 
of moments). However, they can no longer guarantee, as it was the case in this paper, that the existence of 
a controller for the original stochastic control system leads to the existence of a controller for the symbolic 
model. □ 



5.1. Relationship v^rith notions of probabilistic abstractions in the literature. In the remainder of 
this section we relate the (approximate) equivalence relations introduced in this work with known probabilistic 
concepts in the literature. 

The approximate bisimulation notion in Definition |4.2| is structurally different than the probabilistic version 
discussed for finite state, discrete-time labeled Markov chains in jDLTOSj . which is also extended to continuous- 
space processes as in |DEP02| and hinges on the one-step difference between transition probabilities over sets 
covering the state space. The notion in this work can be instead related to the approximate probabilistic 
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bisimulation notion discussed in jJP09| . which lower bounds the probability that the Euclidean distance 
between any trace of abstract model and the corresponding one of the concrete model and vice versa remains 
close: both notions hinge on distances over trajectories, rather than over transition probabilities as in |DEP02I 
IDLTOSj . 

We make the above statement more precise with the following results, which do not require any more the 
assumptions that /(0,i,OTO) ~ 0„ and a{0„) = 0„xp- 

Theorem 5.8. Let T, = (M", \J,Ut, f,<y) be a stochastic control system. Assume that there exists a stochastic 
bisimulation function (f) : M"xM" — >• K^j", as defined in |JP09) . between S and its corresponding non-probabilistic 
control system S = (M", U, Wt, /, 0„xp); and assume that E is S-ISS. For any e S E"*", any triple q = {T,r/,^) 



of quantization parameters, satisfying /i < span{U) and (5.19) (or (5.12) and {5.13)), and for any state run 
(.Xtov of Sr(J^), there exists a state run Xj^^^u o/S'q(S), and vice versa, such that the following relation holds: 

(5.20) P I sup WUoAkr) X^^„^{k)\\ > e \ x^o] < V'^(^^o, ^ro) + e ^ 

Proof. Following the definition of stochastic bisimulation function in |JP09j . for any input curve v G Ur, there 
exists an input curve v G Uj- such that (j) {^x^ovi^x^ov) ^ nonnegative supermartingale |Oks021 Appendix 
C]. Furthermore, using the chosen triple q — {T,ri,iJ.) of quantization parameters, we have: Sr (S) =^ S'q(I]), 
implying that for a state run ^Xtqv of '^r (S), there exists a state run Xx^gu of S'q(S]) such that: 

sup Wlx^avikr) - X^^„„(fc)|| < e. 

fcSNo 



The above statement follows by direct application of Definition |4.2[ Since an increasing concave function of a 
supermartingale is a supermartingale [Yeh95| Theorem 5.11], we have the following chain of (in)equalities: 



IP i sup Mx^ovikr) - Xa;^o„(fc)|| > e \ XrO 

UeNo 



Pi sup WCx^ovikr) - Cx^ovif^'^) +^x^ovif^'^) -^^,ou{k)\\ >e\xro>< 

\keNo 



sup iWCx^o^ikr) - Cx^^^ikT)\\ + WCx^ovi^'^) -'^^qou{k)\\} > e \ Xro> < 

LfcGNo J 

p(sup {{HUo.{kr),-ix^,dkr))r}+s>e\ xj < VH^rO^x^o) + e ^ 
In a similar way, we can prove that for any state run y^x^au of 'S'q(E), there exists a state run S.x^ai' of '5't(E) 



such that the relation in (5.20) holds. □ 



Theorem 3 in |JP09j provides a similar result as in Theorem 5.8 however it is limited to hold over two infinite 
systems rather than a finite and an infinite one as in this work. As explained in ^JP09) . in order to compute 
a stochastic bisimulation function, one requires to solve a game, which is computationally difficult even for 
linear stochastic control systems. On the other hand, if the set of input is a singleton (i.e., U = {Om}), which 
results in the verification of stochastic dynamical systems, then a stochastic bisimulation function (jj can be 
efficiently computed by solving some LMIs, as explained in |JP091 equations (39) and (40)] for linear stochastic 
control systems. Alternatively, the stochastic contractivity of the model can be used for this goal |Aba09| . 

Remark 5.9. Let us further comment on the relationship between 5-LSS-Mq Lyapunov functions in this work 
and stochastic bisimulation ones in |JP09| . Let E = (M", {Om}, ^t, /, o") be a stochastic control system admit- 
ting a 5-ISS-Mq Lyapunov function V, and such that /(0„, 0„i) = 0„, (t(0„) = Onxp, and 0„xn ^ dx,xy{Omx) 
for all X € M". It can be readily verified that the function (j){x,x) — V{x, 0,i) + V^(0„,a;), for all x,x € M", is 
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a stochastic bisimulation function between E and the corresponding non-probabilistic control system under 
the following modification in condition (i) in |JP09[ Definition 2]; (i) (j){x,x) >a{\\x — x\\), for some a € /Coo 
and all x,x £ W . Using this new condition on (p, one can readily revise the relation (5.201 correspondingly. 

□ 



The next theorem, which is computationally more tractable for stochastic control systems and does not require 
the existence of stochastic bisimulation functions as in |JP09j . provides a similar result as in Theorem 5.8 but 
applies only over a finite time horizon. 

Theorem 5.10. Let T, = (M", U,Z//r, /, f) be a stochastic control system. Suppose there exists a 5-ISS-Mq 
Lyapunov function V for S with a form as in (3.5) or as in Lemma 3.7, and that Tr{a{x)'^dx,xV{x, y)a{x)) 
is uniformly upper-bounded by a constant a > 0, for all x,y £ M". For any e € , any triple q — (T,ri,fi) 
of quantization parameters, satisfying fi < span{\J) and (5.191 (or (5.12), and ( 5.13[ ) ), and for any state run 
S.Xtov of Sr{^), there exists a state run X^^q„ o/S'q(S), and vice versa, such that the following relations hold: 



(5.21) 
(5.22) 



P 



P 



sup Mx^oAkT)-K,ou{k)\\>e + e\xro} <l-e ^^^-"^ , if a{e'i) > 

0<k<N 



sup Mx^ovikr) - Xx^uik)\\ > e + e I a;^o ^ < 



0<fe<Af 



- 1) a 
2KQ;(e'?)e^^« 



z/a(e«)< 



a 

2^' 

a 

2k' 



Proof. As argued in the proof of Theorem 5.8 for any state run ^^^ov '^r (^) (notice that here the non- 
probabilistic model is excited with the same input v of S), there exists a state run Xx^qu of S'q(S) such 
that: 

sup Wlx^ovikr) - X^^o„(fc)|| < e. 

fcGNo 

Hence, one obtains the following chain of (in)equalities: 



(5.23) 



sup Mx.rouik'^) - ^x^au{k)\\ > e + e I a;^o 

0<k<N 



sup ll^x^ovikT) ^^^^^^{kr) + C {kr) - Xx,ouik)\\ > e + e | a;^o 

0<fc<W 



< 



sup (11^ (kr) - e (kr) +k XrOv{kT) '^X^qU {k)\\} > e + e \ Xrol < 
0<k<N 



sup 

0<fe<Ar 



[a{\\^x^ovikT) - Cx^ovikT-)\\'')] > aie"^) \ Xro 



< 



sup {V {^x^,,^{kT),^ {kr))} > aie"^) \ Xro> ■ 

0<k<N 



As showed for the J-ISS-Mg Lyapunov function V in the proof of Lemma [3. 7[ we have: 



< 



->^V (^..o..(<),^x.o.W) + 2Tr(a(^,^„„(t))^9,,,\/(^,^„„(t),^,^„J0)^(^..o..W)) 



(5.24) 



a 
2' 



for any v €Ur and any Xr{) G K". Using inequalities (5.231, (5.24 1, and Theorem 1 in |Kus67[ Chapter III], 
one obtains the relations (5.21) and (5.22). In a similar way, we can prove that for any state run (,„ of 
S'q(S), there exists a state run ^x^ro^ of AV(S) such that the relations in (5.21) and (5.22) hold. □ 



As an alternative to the two results above, we now introduce a probabilistic approximate bisimulation relation 
between 5t(S) and S'q(S]) point-wise in time: this relation is sufficient to work with LTL specifications which 
satisfiability can be ascertained at single time instances, such as for next (0) and eventually (<C>). 
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Definition 5.11. Consider two systems 5*^(5]) — (Xt-,X^q,Ut-, ^ and 

T 

S'q(S) = (^q, -''^qOj Uq, ► , Yq , -ffq) , and precisions e G and e G [0, 1] . A relation R C X^- x X^ is said 

to he an (e, 'e)- approximate simulation relation from Sr to Sq, if the following three conditions are satisfied: 

(i) for every Xt-q G X^-Oj there exists Xqo G XqQ with (a;ro,2^qo) G R; 

(ii) for every {xr,Xq) £ R we have V {\\Hr{xr) — Hq{xq)\\ > e} < e; 

(iii) for every {xr^Xq) ^ R we have that Xr x'^ in Sr implies the existence of Xq — ^ x'q in Sq 
satisfying {x'^,x'^ G R. 

A relation R C X^. x Xq is said to he an (e^e)- approximate hisimulation relation between Sr and Sq if R is 
an {e,e) -approximate simulation relation from Sr to Sq and R^^ is an {e,e)- approximate simulation relation 
from Sq to Sr- 

System Sr is {e,e)-approximately simulated hy Sq, or Sq {e, e) -approximately simulates Sr, denoted by Sr ^5'^^ •S'q, 
if there exists an {e,e)- approximate simulation relation from Sr to Sq. System Sr is {e,e)- approximate hisimi- 

lar to Sq, denoted by Sr =^'^'^^ Sq, if there exists an {e,e)- approximate hisimulation relation R between Sr and 

Sq. 

We show next the existence of symboUc models with respect to the notion of approximate hisimulation relation 
in Definition l5.11l 

Theorem 5.12. Let S = (M", U,ZYr, /, cr) be a stochastic control system. For any e G and £ G [0,1], we 
have SriTs) ^^'^ Sq{J:) tf Sr{^) 5q(S) for e = ee. 



Proof. The proof is a simple consequence of Theorems 5.1 or 5.3 and Markov inequality |Oks02) . used as the 
following: 

FUlHriXr) - HqiXq)\\ > e} < n\\Hri.r)~HqiXqm ^ m\\HrM-HqiXq)r]y^ ^e_^ 

□ 

6. Case Studies 

We now experimentally demonstrate the effectiveness of the discussed results. In the examples below, the 
computation of the abstract systems S'q(S) have been implemented by the software tool Pessoa |MDT10) on 
a laptop with CPU 2GHz Intel Core 17. We have assumed that the control inputs are piecewise constant of 
duration r and that lAr is finite and contains curves taking values in [U]^. Hence, as explained in Remark 



5.6 



/i = is to be used in the conditions (5.4) and (5.14). The controllers enforcing the specifications of interest 
have been found by standard algorithms from game theory |MPS95[ ITho95j . as implemented in Pessoa. In 
both examples, the terms Wl,i = 1,2, denote the standard Brownian motion terms. 

6.1. Nonlinear model, 2nd moment, "sequential target tracking" property. Consider the nonlinear 
model of a pendulum on a cart borrowed from jBPD12| . which is now affected by noise. The model is described 
by: 

da -6dt + 0.03adiy/, 

d6 = (-f sin(a) - ^6 + ^.v) dt + 0.036 d 1^,1, 

where ^ a-nd ^ represent the angular position and the velocity of the point mass on the pendulum, v is the 
torque applied to the cart, g = 9.8 is acceleration due to gravity, / = 0.5 is the length of the shaft, m = 0.6 is 
the mass, and A: = 2 is the friction coefficient. All the constants and the variables are considered in SI units. 
We assume that U = [—1.5, 1.5] and that Ur contains curves taking values in [U]o.5. We work on the subset 
D = [—1, 1] X [—1, 1] of the state space of S. Using the Lyapunov function V{x,x') — {x — x')'^P{x — x'), 



(6.1) 
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for all x,x' e M^, proposed in |BPD12| . where P = 



conditions (i) and (ii 
condition (iii) in Definition 3.2 with k 
Therefore, E is 



1.5 0.3 
0.3 1.5 

: 1.2 r, and a{r) — 3.6 r, for q — 
and p{r) = 8.76 r, for all x^x' 



it can be readily verified that V satisfies 
2. 

e 



in Definition 3.2 with a(r) 
\ \ 0.7691 

(5-ISS-M2, equipped with the (5-ISS-M2 Lyapunov function V, as long as we are interested in 

one obtains that functions /3(r, s) 



Moreover, V satisfies 
Z?, and again q — 2. 



its dynamics, initialized on D. Using the results in Theorem 
and j{r) — 3.49 r satisfy property (3.1) for E. 



3.3 



3e- 



For a given fixed sampling time t — 3, the precision e is lower bounded by 0.0778 and 0.4848 using the results 
in Theorems |5.3| and |5.1[ respectively. Hence, the results in Theorem |5.3| provide a symbolic model which is 
much less conservative than the one provided by Theorem 5.1 By selecting an e = 0.085, the parameter jj for 



S'q(I]) based on the results in Theorem 5.3 is obtained as 0.0033. The resulting cardinality of state and input 
sets for 5'q(I]) are 370881 and 7, respectively. The CPU time taken for computing the abstraction S'q(S) has 
amounted to 17278 seconds. 

Now, consider the objective to design a controller forcing the trajectories of S, starting from the initial 
condition xq — (0, 0), to first sequentially visit (in the 2nd moment metric) two regions of interest Wi — 
[0.48, 0.58] x[—l, l],andW2 = [—0.58, — 0.48]x[— 1, 1]; then, once the system has visited the regions, to return 
to the first region Wi and remain there forever. The LTL formulgj^encoding this goal is ODW^i AO {Wi A OVF2). 
The CPU time taken for synthesizing the controller has amounted to 3.02 seconds. Figure [T] displays a few 
realizations of the closed-loop solution process S.xov stemming from the initial condition a;o = (0, 0), as well as 
the corresponding evolution of the input signal. In Figure [2j we show the square root of the average value (over 
100 experiments) of the squared distance in time of the solution process £,xov to the sets Wi and W2, namely 
||Ca;oi'(^)llH'i ^^'^ ll^a:oi^(^)llvK2' '^ti^J'^ the point-to-sct distance is defined as \\x\\w = infmgvK — w\\. Notice 
that the square root of this empirical (averaged) squared distances is significantly lower than the selected 
bound on the precision e = 0.085, as expected since the conditions based on Lyapunov functions can lead to 
conservative bounds. (We have discussed that the bounds can be improved by seeking optimized Lyapunov 
functions.) 



Notice that using the chosen quantization parameters and (5.19), we obtain a precision e ~ 0.005 in (5.21 1 or 



(5.22). Using the result in Theorem 5.10 one can conclude that, as long as we are interested in dynamics of 



S on D, we have an a = 0.0032 in Theorem |5.10[ and that the refinement of a controller satisfying an LTL 
formula ip for S'q(I]) (e.g. ip = OOWi A O {Wi A OW2)) to the system S'r(S) satisfies the "inflated formula" 
(e.g. = AO (W^f^i A OW§-^^)) with probability at least 70% over a discrete time horizon 

spanning {0, 3, • • • , 27} seconds, where 1^9" '^^ is 0.31-infiation of ip as defined in |LT0M12] . 



Furthermore, employing the result in Theorem |5.12[ one can conclude that the refinement of a controller satis- 
fying OW for S'q(E) to system S'^(E) satisfies OW with probability 1-e/e, where W = {xeM." \ \\x\\^ < e}. 
For example, refinement of a controller, satisfying OWi for S'q(S), to the system S'r(S) satisfies OWi'"^^ with 
probability at least 70%. 



(6.2) 



6.2. Linear model, 1st moment, "reach-and-stay, Vifhile staying" property. Consider a linear DC 
motor borrowed from |MT| . now affected by noise and described by: 

da = (-)ei + f^2)d< + 0.15^1 dW^.i, 
d6 = (-Ki-f6 + rt>) di + 0.156 d 1^2^ 

where 6 is the angular velocity of the motor, 6 is the current through an inductor, v is the voltage signal, 
b = 10"'' is the damping ratio of the mechanical system, J = 25 x 10~^ is the moment of inertia of the rotor, 
K — 5 X 10~2 is the electromotive force constant, L = 3 x 10~^ is the electric inductance, and R = 0.5 is 
the electric resistance. All c onsta nts and variables refer to the SI units. It can be readily verified that the 

1.2201 



system E satisfies condition (3.8) with constant k = 40 and matrix P 



0.2224 



0.2224 
1.2248 



Therefore, S 



Note that the semantics of LTL are defined over the output behaviors of S'q(S). 
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Figure 1. Example 1: A few realizations of the closed-loop solution process Cxqv with initial 
condition xq = (0, 0) (bottom panel) and the corresponding evolution of the obtained (unique) 
input signal v (top panel). 



1.5 




qI — , , U , , ^ qI ^U-J ' ' ' ^ 

10 20 30 40 50 10 20 30 40 50 

time time 



Figure 2. Example 1: Square root of the average values (over 100 experiments) of the 
squared distance of the solution process ^xgv to the sets Wi (left panels) and W2 (right 
panels), in two different vertical scales (top vs bottom panels). 



is 5-ISS-M, and equipped with the (5-ISS-Mq Lyapunov function V{x,x') in (3.5), where q £ {1,2}. In this 
example, we use q = 1- 

We assume that U = [—0.5, 0.5] and that Ur contains curves taking values in [U]o.i. We work on the subset 

D = [—5, 5] X [—5, 5] of the state space of E. For a given precision e = 1 and fixed sampling time r = 0.01, 

the parameter rj of >S'q(E) based on the results in Theorem 5.1 is equal to 0.01. Note that for sampling times 

T < 0.026, and in particular for a cho ice r = 0.01, the results in Theorem 15.31 cannot be applied here because 

1 J I I I ' ' 

(/? (e*^, t)) « > e and condition (5.14) in Theorem 5.3 is not fulfilled. The resulting cardinality of the state 

and input sets for S'q(I]) amounts to 1002001 and 11, respectively. The CPU time used for computing the 

abstraction has amounted to 148.092 seconds. 
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Now, consider the objective to design a controller forcing (in the 1st moment metric) the trajectories of E 
to reach and stay within W = [4.5, 5] x [—0.25, 0.25], while ensuring that the current through the inductor 
is restricted between —0.25 and 0.25. This corresponds to the LTL specification ODT^/' A OZ, where Z = 
[—5, 5] X [—0.25, 0.25]. The CPU time used for synthesizing the controller has been of 3.88 seconds. Figure 
[3] displays a few realizations of the closed-loop solution process ^xqv stemming from the initial condition 
Xq — (—5, —0), as well as the corresponding evolution of the input signal. In Figure [4j we show the average 
value (over 100 experiments) of the distance in time of the solution process ^xgv to the sets W and Z, namely 
ll^a:oi;(i)llH' ^^'^ Mxov{t)\\z- Notice that the empirical average distances are as expected significantly lower 
than the precision e = 1. 



Note that using the selected quantization parameters in (5.12) and (5.13), we obtain a value e = 0.08 in (5.20), 



(5.21) or (5.22). Further, using the result in Theorem 5.8 one obtains that, as long as we are interested in 
dynamics of E, initialized on D and U = {Om}, there exists a state run y-x^o^ of '5'q(S) satisfying an LTL 
formula (p if and only if there exists a state run £,x^qv of S't(E) satisfying ip^ with probability at least 79% over 
the infinite time horizon, where (p^ is the 1-inflation of (p. 



Moreover, using the result in Theorem |5.10[ one can conclude that, as long as we are interested in dynamics of 
S on Z, implying that a = 0.6917 in Theorem |5.10[ the refinement of a controller, satisfying an LTL formula 
ip for S'q(S) (e.g. if = OnWAOZ) to the system Sr{^) satisfies ip^ (e.g. ip^ = OnW^ A DZ^) with probability 
at least 70% over a discrete time horizon spanning {0,0.01,0.02, • • • ,1} seconds. 







































w 







1 

time 



Figure 3. Example 2: A few realizations of the closed-loop solution process £,xov with initial 
condition xq = (—5, 0) (left panel), and the evolution of the input signal v (right panel). 



7. Conclusions 

This work has shown that any stochastic sampled-data control system, admitting a (5-ISS-Mg Lyapunov func- 



tion of the form in (3.5) or with a shape as in Lemma 3.7 and initializing within a compact set of states, 
admits a finite approximately bisimilar symbolic model (in the sense of moments or probability). The con- 
structed symbolic model can be used to synthesize controllers enforcing complex logic specifications, expressed 
via linear temporal logic or as automata on infinite strings. 

The main limitation of the design methodology developed in this paper lies in the cardinality of the set of 
states of the computed symbolic model. The authors are currently investigating several different techniques to 
address this limitation. Promising work over non-probabilistic control systems includes specification-guided 
abstractions |RMT12j . results on differentially flat systems jCDll) . and the use of non-uniform quantization 
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time time 

Figure 4. Example 2: Average values (over 100 experiments) of the distance of the solution 
process £,xov to the sets W (left panels) and Z (right panels), in two different vertical scales 
(top vs bottom panels). 

[TI09| ■ Furthermore, the authors are currently working toward extensions of the results over general stochastic 
hybrid systems. 
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9. Appendix 



3.2 



Proof of L emm a \3.J\ It is not difficult to check that the function V in (3.51 satisfies properties (i) and (ii) of 
Definition 



with functions a{y) :— ^^Amin {P)^ ' V and a{y) (^^Amax {P) ) ' V- It then suffices to verify 



property (iii). We verify property (iii) for the case that / is differentiable and using condition (3.7). The 
proof, using condition (3.61, follows similarly by removing the inequalities in the proof including derivative of 



/. By the definition of V in (3.5), for any x,x' e E" such that x ^ x', and for q e {1, 2}, one has 



d,^y = ~d,^y ^{x- x'Y P V{x, x') 



dr.V = d^,,^,V = -d^,^,V ^P 



{V{x, x')) ' ' + "^Pix - x')[x - x'fP (V{x, x')) ■ 



Therefore, following the definition of , and for any x,x' , z G M" such that x x' , and any u,u' ^ U, one 



obtains the chain of (in)equalities in (9.1). In (9.1), z e E" and the mean value theorem [DK04) is applied to 
the differentiable function x i— f{x,u) at points x,x' for a given input value u € U and is the Lipschitz 



3.2 



con stant, as introduced in Definition 2.1 Therefore, the function V in (3.5 ) satisfies property (iii) of Definition 

■ g 



with positive constant — ^ and /Coo function p{r) 



□ 



Proof of Lemma \37^ In the proof, we use the notation aa'^{x) instead of a{x)a'^{x) and H{V){x, x') for the 
Hessian matrix of V at S E^". We drop the arguments of dx.xV , dxV, dx'V, and H{V) for the sake of 



(9.1) 



'a{x) 



(y^ [x] (T^(a;')j 



dx,xV -dx,xV 
-dx,xV dx,xV 



"V(a;, x') = {x- x'fP (V{x, x')) ' ' {fix, u) - fix', u')) + ^1 

= ix- x'fP {Vix, x')) {fix, u) ~ fix', u')) + ^Tr ((a(a;) - aix')) [a^ {x) - a^(x')) dx,xV) 
= ix- x'fP (vix, x')) '''' {fix, u) - fix', u')) + \ l^ioix) ~- oix')) {yix, x')) 

+ j|(a;-x-yP(o-(a;) -o-(x-'))||^ (\/(x, x')) 

<ix- x'fP (vix, x')) '''' {fix, u) - fix', u) + fix', u) - fix',u')) 

+ \\^{^ix)-aix'))\^^(vix,x')f'' 



<ix- x'yP [Vix, x')j dxfiz, u)ix - x') + ix- x'yP [Vix, x')j {fix',u) - fix', u')) 
+ ^\\^/Piaix)-aix'))f^(vix,x') 



<{ix-x') Pdxfiz,u)ix- 



)ix - x') + 1 ll^/P (a(x) - a(x'))|Q [vix, x')) 



+ ^[ix-x')^Pix-x')y |\/p|l„||u-u'|| {yix,x')Y' 
, , ,, nsL?, II ^119 ,, 
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simplicity. In view of Ito's formula, Jensen's inequality, and similar to calculations in Lemma |3.4[ we have 



/' 

Jo 

f 

Jo 



[d^V d,,V] 
[d^V d^,V] 



ds 



-Tr 



o-i£.xvis}) 



<T^&.W) ^^U:.^(S)) H{V) 



(9.2) 



(9.3) 



/' 



■f 

Jo 



i Tr (fja^ (C^^,(s))9a;,:.v) " ^ Tr 



a{i,xv{s)) 



ds 



<r'^ {^x.{s)) CT-^ U^^(s)]\H(V) 



ds 



f 

Jo 



< / -«E y(c,„(s),5,js 



:Tr <7f7^ {£,:,,,{s))d^,^V] 



ds < h{cr, t)e 



ds. Inequality (9.2 1 is 



where the function h can be computed as h{t,(T) := Jp* E ^ || \/5zvrl^cr 

a straightforward consequence of V satisfying the condition (iii) in Definition [3.2[ and (9.3 1 follows from 
Gronwall's inequality. Using the Lipschitz continuity assumption on the diffusion term a, we get: 





'1 






Jo 


2 



ds < - sup ■ 



dx,xV{x,x') 



nmm{n,p}Z'^ / E \\^xv{ 



ds. 



Since y is a (5-ISS-Mg Lyapunov function, q>2, /(0„, Om) = 0„, ^(On) — 0„xp, and using functions P and 7 
in (3.1 1, one can verify that: 



E 



\ixvm\ 



< E 



^mxAt)\n<HM',t)+^i\\v\ 



and hence 



Therefore, we have 



E 



<l3(snp{\\xf},t) +7fsup{||u||}) 



UxAt)f < sup{||.T||''},0 +7 sup {11^.11} 



xeD 



Ht, fj) < - sup ■ 
^ x.x'eD 

By defining: 
(9.4) 

/i((T, t) — a~ 
we obtain E 

conditions of the lemma. 



dx,xV{x,x') 



rimin{n,p}Z / /3 sup } , s + 7 sup {||u||} ds 



x£D 



sup 

x.x'^D 



\^dx,xV{x,x')\\ \nmm{n,p}Z 



72 -Kt 

e 



a-6D 



/3 sup{|lxr},s +7 sup{|ltt|l} , 



lieu 



— < h{<7,t). It is not hard to observe that the proposed function h meets the 

□ 
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Proof of Lemma \3.S\ In the proof, we use the notation aa'^ (x) instead of a{x)a'^ (x) for the sake of simphcity. 
In view of Ito's formula and similar to calculations in Lemma 13.71 we have 



(9.5) 



(9.6) 



^A„,in(P)E[||e..(t)-C.Ji)f 



- 2 



= / E 



= / E 



+ i Tr {aa^ (^„,(s) - C,,(=s)) Z') + ^ Tr {aa^ (^,„(s)) P - aa^ (e„,(s) - ^,„(s)) P) 



ds 



ds 



< / -kE 
Jo 



ds 



^ E [Tr {aa^ (^^s)) P - aa^ {^^s) - l,,{s)) P)] ds 



< 



-kE 



{^xv{s)A^.{s))\ds + - M[TT{aa^{^,Js))P)]ds<h{t,a)e^ 



where the function h can be computed as h{t, a) :— ^J* E 



ds. Inequality (9.5 1 is a straight 



forward consequence of (3.6), and (9.6) follows from Gronwall's inequality. Using the Lipschitz continuity 



assumption on the diffusion term a, we get: 



2 7o 



E 



ds < 



^min{n,p}Z^ / E \\^xv{ 



ds. 



Sinc e V i is a 5-ISS-M2 Lyapunov function, /(0„,0m) = 0„, cr(0„) = 0„xp, and using functions /3 and 7 in 

I 2 
(3.1), obtained by the o-ISS-M^ Lyapunov function V 1 , one can verify that: 



E 



2n 



eK2Amin(f') 



Therefore, one obtains: 



h{t,a) < 



•n? min{n.p}Z^ 

Amin(-P)K 



A,nax(i") sup{||a:|p} + 



— sup t 



By defining: 
(9.7) 

c 

h{a,t) =- 
we obtain E 

conditions of the lemma. 



min{7i,p}Z^e 1 ^ 



'^miniP)^ 



dp){ 



\ 



1 -e^ ) sup {||a;|p} + 
^ xeD 



Li 



sup{\\u\\^}t , 



^xvit) — Cxv{^)\\ — h{a-,t). It is not hard to observe that the proposed function h meets the 

□ 
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Proof of Corollary \3.1C\ Motivated by inequality (9.5), one can obtain 
E [Tr {oa^ P ~ aa^ {i^s) - P)\ 

where satisfies the ODE ^^.^(i) = ^Cxu(^) + Bv{t). It can be readily verified that 
(9-8) \\lxv{-t)\\<h^'\\\\x\\ + { I ||e^^B||ds) llulloo < ||e^*|| sup{||a;||}+ f / ||e^^S|| ds) sup {||m||} 



tieu 



The above approximation, together with (9.5 ), leads to a more explicit bound in terms of the system parameters 

as follows: 

(9.9) 



E 



< 



E ^TP<^^ ) e- 

1^1 



'^min (P) 



le^^W sup {||x||} - 



where k - 



2k 



\e^''B\\dr ] sup {\\u\\}] ds, 

□ 
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